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Abstract 


We  consider  two  player  electromagnetic  evasion-pursuit  games  where  each  player 
must  incorporate  significant  uncertainty  into  their  design  strategies  to  disguise  their 
intension  and  confuse  their  opponent.  In  this  paper,  the  evader  is  allowed  to  make 
dynamic  changes  to  his  strategies  in  response  to  the  dynamic  input  with  uncertainty 
from  the  interrogator.  The  problem  is  formulated  in  two  different  ways;  one  is  based 
on  the  evolution  of  the  probability  density  function  of  the  intensity  of  reflected  signal 
and  leads  to  a  controlled  forward  Kolmogorov  or  Fokker-Planck  equation.  The  other 
formulation  is  based  on  the  evolution  of  expected  value  of  the  intensity  of  reflected 
signal  and  leads  to  controlled  backward  Kolmogorov  equations.  In  addition,  a  number 
of  numerical  results  are  presented  to  illustrate  the  usefulness  of  the  proposed  approach 
in  exploring  problems  of  control  in  a  general  dynamic  game  setting. 
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1  Introduction 


In  an  electromagnetic  evasion-interrogation  game,  the  evader  wishes  to  minimize  the  in¬ 
tensity  of  the  reflected  signal  to  remain  undetected  in  carrying  out  his  mission  while  the 
interrogator  wishes  to  maximize  the  intensity  of  reflected  signal  to  detect  the  attacker.  It 
was  demonstrated  in  [9]  that  it  is  possible  to  design  ferroelectric  materials  with  appropri¬ 
ate  dielectric  permittivity  and  magnetic  permeability  to  significantly  attenuate  reflections 
of  electromagnetic  interrogation  signals  from  highly  conductive  targets  such  as  airfoils  and 
missiles.  These  results  were  further  sharpened  and  illustrated  in  [10]  where  a  series  of  differ¬ 
ent  material  designs  were  considered  to  minimize  over  a  given  set  of  input  design  frequencies 
the  maximum  reflected  held  from  input  signals.  In  addition,  it  was  shown  that  if  the  evader 
employed  a  simple  counter  interrogation  design  based  on  a  fixed  set  (assumed  known)  of 
interrogating  frequencies,  then  by  a  rather  simple  counter-counter  interrogation  strategy 
(use  of  an  interrogating  frequency  little  more  than  10%  different  from  the  assumed  design 
frequencies),  the  interrogator  can  easily  defeat  the  evader’s  material  coatings  counter  inter¬ 
rogation  strategy  to  obtain  strong  reflected  signals.  From  the  combined  results  of  [9,  10]  it 
is  thus  rather  easily  concluded  that  the  evader  and  the  interrogator  must  each  try  to  confuse 
the  other  by  introducing  significant  uncertainty  in  their  design  and  interrogating  strategies, 
respectively. 

Static  two-player  non-cooperative  games  with  uncertainty  were  considered  in  [6].  In  these 
problems,  the  evader  and  the  interrogator  are  each  subject  to  uncertainties  as  to  the  actions 
of  the  other.  The  evader  wants  to  choose  a  best  coating  design  (i.e.,  best  dielectric  permit¬ 
tivities  and  magnetic  permeabilities)  while  the  interrogator  wants  to  choose  best  angles  of 
interrogation  and  interrogating  frequencies  for  input  signals.  Each  player  must  act  in  the 
presence  of  incomplete  information  about  the  other’s  action.  Partial  information  regarding 
capabilities  and  tendencies  of  the  adversary  can  be  embodied  in  probability  distributions  for 
the  choices  to  be  made.  That  is,  one  may  formalize  this  by  assuming  the  evader  may  choose 
(with  an  as  yet  to  be  determined  set  of  probabilities)  dielectric  permittivity  and  magnetic 
permeability  parameters  from  given  admissible  sets  while  the  interrogator  chooses  angles  of 
interrogation  and  interrogating  frequencies  from  appropriate  admissible  sets  .  The  formu¬ 
lation  in  [6]  is  based  on  the  mixed  strategies  proposals  of  von  Neumann  [2,  30,  31]  and  the 
ideas  can  be  summarized  as  follows.  The  evader  does  not  choose  a  single  coating,  but  rather 
has  a  set  of  possibilities  available  for  choice.  He  only  chooses  the  probabilities  with  which 
he  will  employ  the  materials  on  a  target.  This,  in  effect,  disguises  his  intentions  from  his 
adversary.  By  choosing  his  coatings  randomly  (according  to  a  best  strategy  to  be  determined 
in,  for  example,  a  minrnax  game),  he  prevents  adversaries  from  discovering  which  coating  he 
will  use-indeed,  even  he  does  not  know  which  coating  will  be  chosen  for  a  given  target.  The 
interrogator,  in  a  similar  approach,  determines  best  probabilities  for  choices  of  frequency 
and  angle  in  the  interrogating  signals.  Note  that  such  a  formulation  tacitly  assumes  that 
the  adversarial  relationship  persists  with  multiple  attempts  at  evasion  and  detection. 

The  problems  are  mathematically  formulated  in  [6]  as  two  sided  optimization  problems  over 
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spaces  of  probability  measures,  i.e.,  minmax  games  over  sets  of  probability  measures.  That 
work  demonstrates  the  feasibility  and  the  potential  usefulness  of  developing  theories  for  prob¬ 
lems  with  uncertainty.  In  this  paper,  we  move  toward  a  more  realistic  dynamic  modeling 
by  introducing  time  dynamics  into  the  problem  for  single  evasion  attempts.  Specifically,  we 
allow  a  single  evader  to  make  dynamic  changes  to  his  dielectric  permittivity  strategies  in 
response  to  feedback  entailing  measures  of  the  reflection  signals  based  on  dynamic  informa¬ 
tion  with  uncertainty  about  the  interrogator’s  choices.  Thus,  this  new  formulation  is  more 
in  the  spirit  of  the  deterministic  dynamical  differential  games  as  formulated,  for  example,  in 
[20]  except  here  uncertainties  of  the  two  players’  actions  are  a  major  feature  as  in  the  static 
games  of  [6,  31].  The  remainder  of  this  paper  is  organized  as  follows.  We  begin  in  Section  2 
by  presenting  a  description  of  our  problem  formulation.  We  then  outline  a  theoretical  and 
computational  framework  in  Section  3  that  provides  a  foundation  for  our  solution  methods. 
A  number  of  computational  results  are  presented  in  Section  4.  We  conclude  the  paper  by 
some  summary  remarks  and  proposed  future  research  efforts  in  Section  5. 


2  Problem  Description 


In  this  section,  we  will  use  a  capital  italic  letter  to  denote  a  random  variable  unless  otherwise 
indicated,  and  use  the  corresponding  small  letter  to  denote  its  realization. 

We  formulate  a  minimization  problem  with  cost  functional  in  terms  of  some  reflection  coeffi¬ 
cient  dependent  on  the  evader’s  dielectric  permittivity  e  as  well  as  the  interrogator  frequency 
u.  This  reflection  coefficient  could  be  based  on  a  simple  planar  geometry  using  Fresnel’s  for¬ 
mula  for  a  perfectly  conducting  half  plane  which  has  a  coating  layer  of  thickness  d  with 
dielectric  permittivity  e  and  interrogating  frequency  oo  as  detailed  in  [6].  This  expression  can 
be  derived  directly  from  Maxwell’s  equation  by  considering  the  ratio  of  reflected  to  incident 
wave  for  example  in  the  case  of  parallel  polarized  ( TEX )  incident  wave  (see  [9,  25]). 

An  alternative  and  much  more  computationally  intensive  approach,  which  may  be  necessi¬ 
tated  by  some  target  geometries  (e.g.,  missiles),  employs  the  far  held  pattern  for  reflected 
waves  computed  directly  using  Maxwell’s  equations.  In  two  dimensions,  for  a  reflecting 
body  with  a  given  coating  layer  with  an  interrogating  plane  wave  E^\  the  scattered  held 
E ^  satishes  the  Helmholtz  equation  [18]  as  detailed  in  [6]. 

Throughout  we  assume  for  simplicity  that  the  magnetic  permeability  for  the  evader  is  hxed 
as  is  the  angle  of  incidence  of  the  interrogating  signal.  We  assume  that  the  evader  has 
the  ability  to  choose  the  dielectric  parameter  e  he  uses  in  order  to  thwart  detection,  and 
the  parameter  e  is  changed  adaptively  depending  on  the  frequency  w  that  the  interrogator 
is  using  (or  rather  depending  on  the  rehections  produced  by  the  interrogator’s  frequency 
choices).  In  addition,  we  assume  that  the  interrogator  frequency  process  { Wt  :  t  >  0}  is  an 
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Ito  diffusion  process  (Chapter  7  of  [29])  satisfying  the  stochastic  differential  equation 

dWt  =  n{Wt)dt  +  a(Wt)dBt,  (2.1) 

where  both  p,  and  a  are  non-random  functions  that  are  Lipschitz  continuous,  and  Bt  denotes 
the  standard  Brownian  motion. 

Below  we  will  consider  two  different  formulations,  one  is  based  on  the  evolution  of  the 
probability  density  function  of  the  intensity  of  reflected  signal,  and  the  other  is  based  on  the 
evolution  of  expected  value  of  the  intensity  of  reflected  signal. 


2.1  Evolution  of  Probability  Density  Function  of  Intensity 


Let  Yt  =  tp(Wt),  where  p  is  some  chosen  measure  of  intensity  of  the  reflected  signal  for  a 
given  material  dielectric  parameter  (for  example,  p  can  be  chosen  as  the  magnitude  of  the 
reflection  coefficient).  In  addition,  we  assume  that  p  is  twice  continuously  differentiable. 
Then  by  Ito’s  formula  we  find  that 


dYt 


p\Wt)p(Wt)  +  \p"{Wt)o2{Wt) 


dt  +  p\Wt)a{Wt)dBt. 


If  we  further  assume  that  p  1  exists,  then  we  can  rewrite  the  right-hand  side  of  the  above 
equation  in  terms  of  Yt  given  by 


dYt 


p\p  1(Yt))p(p  1(Yt))  +  ^<p"(<p  1(Yt))a2(p  x(yt)) 
+p\p~1(Yt))a(p~1(Yt))dBt. 


dt 


If  p  is  chosen  such  that  functions  p\p  1(y))p(p  1(y))  +  ~p"{p  1(y))&2(p  1(y))  and 

p' (p~l(y))a(p~l(y))  are  both  Lipschitz  continuous,  then  {Yt  :  t  >  0}  is  also  an  Ito  diffusion 
process.  Let  p(t,y)  denote  the  probability  density  function  of  the  random  variable  Yt.  Then 
it  is  well  known  that  p  satisfies  Fokker-Planck  equation  (e.g.,  see  [23,  p.  118]) 


dp(t,y)  d_ 

dt  dy 


l{v))  +  x( y ))^ 


1  d2 

2  dy* 


l(y))°{v  l{y)))2 pit, y) 


p(t,y) 


(2.2) 


For  our  illustration  here,  we  choose  p  to  be  a  first-order  approximation  of  the  chosen  measure 
of  intensity  of  the  reflected  signal,  that  is,  p{w )  =  CiW  +  c0  (i.e.,  y  =  citc  +  c0),  where  c0  and 
Ci  are  constants.  Then  (2.2)  can  be  simplified  as 


dPfc y)  +  \v))pit ,  y)]  =  1(?/))p(^  y)]  ■  (2-3) 
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Let  e(t,w)  =  p(t,y).  Then  we  have 

dg  dp  d  Id  d 2 

dt  dt1  dy  ci  dw  ’  dy2 

Hence,  we  can  rewrite  (2.3)  in  terms  of  w  as  follows 


1  d2 
c\  dw2 ' 


+  iL  b*(w)e(t,w)]  =  [a2(w)g{t,w)]  . 


dt 


2  dw 2 


(2.4) 


To  allow  for  evader  control  of  the  system  (2.4),  we  introduce  some  input  of  the  form 

—\r(g(t,  w)  —  u(t,  e)) 

into  the  system  (2.4),  that  is, 

f)o(t  vA  3  1  (ft 

— df  +  dw  w =  2  dw2  w )]  _  xr(e(t,  w)  -  u(t,  e)),  (2.5) 

where  Ar  is  the  relaxation  constant  of  the  material.  We  note  from  (2.5)  that  once  we  introduce 
feedback  controls  into  this  system,  g  is  no  longer  a  probability  density  function  (indeed  we 
are  trying  to  drive  it  to  zero  on  most  of  its  support). 

Here  we  consider  a  generalized  control  u,  which  is  defined  by 

u(t,  w)  —  J  r(w,e)dU(t,e),  (2.6) 

where  r(w,  e)  is  some  given  real- valued  function  of  the  reflection  coefficient  with  given  fre¬ 
quency  w  and  dielectric  parameter  e,  and  U  is  a  time-dependent  distribution  of  possible 
dielectric  settings  e  in  S.  The  motivation  for  introducing  distributional  or  generalized  con¬ 
trols  is  two  fold.  First,  this  is  natural  when  one  is  extending  the  static  theory  of  [6]  where 
the  uncertainty  in  controls  is  embodied  in  probability  measures  on  the  static  control  param¬ 
eters  such  as  dielectric  permittivities  and  interrogating  frequencies.  A  second  compelling 
motivation  is  prompted  by  a  rich  literature  on  closure  theorems  in  the  calculus  of  variations 
and  optimal  control  associated  with  distinguished  contributors  such  as  Young  [37,  38],  Mc- 
Shane  [26,  27,  28],  Filippov  [21],  and  Warga  [33,  34,  35],  among  others.  In  some  variational 
and  control  problems  (and  especially  in  two  player  differential  games-see  for  example  the 
discussions  in  [20]  and  the  counter  example  of  Berkovitz  [16]),  it  has  been  known  since  the 
years  of  L.C.  Young  that  one  must  often  introduce  generalized  or  relaxed  controls  (also  called 
sliding  regimes  [21]  or  chattering  controls)  in  order  to  obtain  well  posed  optimization  prob¬ 
lems.  In  anticipation  of  treating  these  two  player  dynamical  games  where  both  the  evader 
and  interrogator  have  time  dependent  controllers,  here  we  use  generalized  controls  for  the 
evader  which  thus  introduces  uncertainty  in  the  evader  controls  as  well  as  uncertainty  in  the 
interrogation  frequencies  via  the  stochastic  dynamics  (2.1). 

We  remark  that  the  use  of  generalized  controls  has  arisen  naturally  in  a  number  of  other 
modern  applications  including  in  smart  materials  with  smoothed  Preisach  controls  [12,  13,  14] 
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where  hysteretic  control  influence  operators  representing  smart  material  actuators  can  be 
used  to  guarantee  well  posedness  as  well  as  to  develop  efficient  computational  algorithms. 


One  of  the  main  benefits  of  relaxed  controls  is  that  the  optimal  relaxed  controller  can  be  ap¬ 
proximated  by  “real  controls”  and  we  shall  do  that  here.  Indeed,  for  computational  purposes, 
we  will  approximate  the  control  u(t,w )  with  delta  approximations  (rigorous  justification  for 
such  approximations  in  the  context  of  the  Prohorov  [32]  or  weak*  (metric)  topology  on  spaces 
of  probability  distributions  [4,  IT,  24]  can  be  found  in  [3,  4,  5]  as  well  as  in  the  closure  the¬ 
ories  from  [26,  27,  28,  33,  34,  35,  37,  38].  We  restrict  the  set  £  to  a  finite  set  (ey }y=r,  thus 
obtaining  the  collection  of  choices  of  materials  available  to  the  evader 

M 

U{t,e)  «  5Zej(t)Ae.(e), 

3= 1 
M 

dU(t,e)  ~  5^(t)<$e.(e)de, 

3= 1 


where  6j(t)  denotes  the  time-dependent  weighings  for  the  material  with  dielectric  permit¬ 
tivity  e*  that  the  evader  may  choose,  and  A  and  5  are  the  Dirac  distribution  and  density, 
respectively.  Let  bj(w)  =  r(w ,  e*),  j  —  1,  2, ... ,  M.  Then  (2.6)  can  be  rewritten  as 


u(t,  w) 


I 


M  M 

r(w,  e)  Y  e3(t)fi*j(e)de  =  Y  e*j )  =  6Me(f), 

j= 1  i=1 


(2.7) 


T 

where  b(w)  =  (bi(w),  b2(w), . . . ,  bM\w ))  and  e(t)  =  (ei(f),  e2 (t), ,  cm(d)  •  Thus,  using  the 
control  (2.7)  we  can  write  (2.5)  as 


’-^  +  1 1 . XM-Sk> 


a2(w)g(t,w)  —X  r(g(t,w)  —  b(w)e(t)),  (2.8) 


which  is  a  controlled  forward  Kolmogorov  or  Fokker-Planck  equation  [1,  22],  A  reason¬ 
able  linear  quadratic  regulator  (LQR)  control  problem  might  involve  minimizing  the  cost 
functional  __ 

oo  w  oo 


J{e) 


\g(tj  w)\2dwdt  +  /  /3\e(t)\2dt, 


0  w 


(2.9) 


subject  to  (2.8).  Here  [w,  w\  is  the  admissible  range  of  interrogator  frequencies. 


2.2  Evolution  of  Expected  Value  of  Intensity 


We  next  discuss  an  alternative  formulation  for  our  problem.  Assuming  that  Wt  satisfies  (2.1) 
and  for  a  given  the  material  dielectric  parameter  value  et  at  time  t,  we  define 

r  ft  i 


v(t,  w)  =  E 


\eXsf(Ws,es)ds  +  v0(Wt)  \  W0  =  w 


Do 
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where  E[  •  |  •  ]  denotes  the  conditional  expectation,  r(w,  e)  again  represents  some  scalar 
valued  controlled  intensity  for  the  reflected  signal  (e.g.,  the  magnitude  of  the  reflection 
coefficient)  depending  on  incoming  frequency  w  and  dielectric  parameter  e,  and  A  >  0  is 
now  a  discount  parameter.  Following  a  standard  technique  [29,  Section  10.3]  for  treating 
integrals,  we  next  define 

Zt=  [  \eXsr(Ws,  es)  ds. 

Jo 

Then  the  process  Xt  =  (Wt,  Zt)T  satisfies 

,,  (  W,  \_(  p(l Vt)  \  (  a(W,)  \ 

A  Z,  )  ~  {  \ex‘f(  W„e,)  )dt+{o  )iBt 

and 

v(t,w)  =  g(t,(w,  0))  for  g(t,  (w,  z))  =  E[Zt  +  v0(Wt)  \  X0  =  (w,  z)T}. 

Here  the  generator  of  the  Ito  diffusion  process  {Xt  :  t  >  0}  is 

d  Id2  d 

C<j>(w,  z)  =  fi(w)  z)  +  ~(t2(w)  Q-ji f>(w ,  z)  +  A extr(w,  et)—<j)(w,  z). 

It  then  follows  from  Section  8.1  in  [29]  that  g  satisfies  the  backward  Kolmogorov  equation 

d 


dt 


g  =  £g,  9(0,  (w,z))  =  z  +  v0(w). 


(2.10) 


A  discussion  of  the  relationship  between  this  state  and  the  semigroup  generated  by  C  can 
be  found  in  [19].  Since  g  =  v  +  z  is  the  solution  to  (2.10),  it  follows  that  v  satisfies 

d  d  1  d2 

— u(t,  w )  =  n{w)  Q^v(t,  w )  +  -a2(w)  w )  +  A extr(w,  et), 

v( 0,  w)  =  Vq(w). 

Now  let  v(t,w )  =  e~xtv(t,w).  It  is  easy  to  show  that  v  satisfies 


d  did 2 

—v(t,  w )  =  n(w)  w)  +  2a2(w)  q^2‘ w)  +  X  (r(u’’  e*)  _  w)) 

u(0,  w)  =  Vq(w). 

We  note  that  the  state  v  in  this  formulation  is 

rt 


(2.11) 


v(t,  w)  =  E 


Ae-A(t“a)f(Ws,  es)  ds  +  e~xtv0(Wt )  |  W0  =  w 


IJ  o 


the  expected  value  of  a  measure  of  the  reflected  intensity. 


In  this  formulation,  the  controlled  reflection  index  r(w,  e)  can  be  extended  to  generalized 
controls  as  in  (2.6),  (2.7)  where  r  =  f.  Thus  we  can  rewrite  equation  (2.11)  using  the 
generalized  control  from  equation  (2.7)  as  follows 


d  d  l  0  d2 

—v(t,  w )  =  g(w)  - —v(t,  w )  +  -o2(w)  w)  -  A  (v(t,  w)  -  b{w)e(t)). 


(2.12) 
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Note  that  the  control  for  this  formulation  is  the  same  as  the  control  for  the  formulation  in 
Section  2.1  although  this  is  a  controlled  backward  Kolmogorov  equation  [1,  22],  The  primary 
difference  is  that  for  this  latter  formulation  the  control  was  naturally  a  part  of  the  dynamics 
equation  and  we  did  not  have  to  artificially  introduce  a  control  into  the  system  as  we  did 
in  Section  2.1.  Thus,  this  second  formulation  is  somewhat  more  direct  and  hence  perhaps 
more  desirable  from  an  intuitive  perspective.  Moreover,  the  backward  Kolmogorov  equation 
of  this  formulation  usually  presents  less  formidable  computational  challenges. 


2.3  Special  Case 

For  our  presentation  of  theoretical,  approximation  and  computational  results  for  the  above 
two  control  problems,  we  consider  without  loss  of  generality  a  special  case  where  the  above 
problems  are  the  same.  Specifically,  from  (2.8)  and  (2.12),  we  see  that  if  we  choose  n  =  0 
and  a  to  be  a  positive  constant  function,  then  (2.8)  and  (2.12)  are  reduced  to  the  same 
controlled  system,  a  controlled  diffusion  equation.  The  choice  of  n  —  constant  >  0  is  a  more 
physically  relevant  case  (resulting  in  a  convection-diffusion  equation  with  either  positive  or 
negative  convective  flow  depending  on  which  dynamics  are  chosen  from  above),  but  all  that 
we  present  below  applies  to  these  cases  albeit  with  more  technical  detail  in  the  theoretical  and 
approximation  frameworks.  In  particular,  the  inclusion  of  the  convective  term  can  greatly 
complicate  the  computational  problem  in  the  forward  Kolmogorov  formulation.  Thus,  for 
our  demonstration  purposes  here  and  for  the  sake  of  simplicity,  we  will  consider  the  reduced 
controlled  system  given  by 


dv 

dt 


d2v 

'ndw~2 


\{v  —  b(w)e{t )) 


(2.13) 


where  rj  =  -cr2  >  0.  We  further  suppose  that  the  frequencies  that  the  interrogator  is  capable 

of  transmitting  are  in  the  range  of  [w,  w\,  i.e.,  the  support  of  the  interrogator  probability 
density  is  finite.  Then  the  boundary  and  initial  conditions  are  given  respectively  by 


v(t,w)  —  v(t,w)  —  0 
v(0,w)  =  Vq(w). 


(2.14) 


In  the  context  of  the  LQR  control  problem  we  thus  must  minimize  the  cost  function 


J(e) 


oo  w 


v(t,  w)\2dwdt  + 


J  P\e(t)\2dt, 


0  w 


0 


(2.15) 


subject  to  (2.13)  and  (2.14).  We  remark  here  that  (3  is  a  design  parameter  which  is  chosen 
to  balance  the  relative  merits  of  reduction  of  reflection  intensity  versus  control  costs  in  the 
control  objectives. 


3  Sesquilinear  Forms:  Theory  and  Numerical  Approx¬ 
imations 


A  fundamental  framework  for  theory,  approximation  and  computation  for  (2.15)  subject  to 
(2. 13)- (2. 14)  is  available  in  the  context  of  an  abstract  control  problem  as  developed  in  [7,  8] 
with  a  accessible  summary  given  in  [15].  For  convenience  and  following  standard  conventions, 
we  use  an  over  dot  ( ' )  to  denote  the  derivative  with  respect  to  the  time  variable  t.  While  the 
results  presented  below  can  readily  be  given  for  both  general  Kolmogorov  formulations,  for 
brevity  we  only  consider  the  canonical  case  described  in  the  previous  section  where  n  =  0. 


3.1  Sesquilinear  Forms 


We  first  present  theoretical  underpinnings  for  our  control  calculations  in  a  real  Hilbert  space 
setting.  All  of  the  results  given  here  are  summarized  in  more  detail  in  [15].  Let  the  Hilbert 
spaces  H  and  V  be  defined  by  H  =  L2(w,uJ )  and  V  =  H(]  (w,  w) .  We  denote  the  topological 
dual  space  of  V  by  V*  =  H~l(w_,w).  If  we  identify  H  with  its  dual  H*  then  V  H  = 
H*  V*  is  a  Gelfand  triple  [15,  36]. 


Define  the  linear  operator  A  :  V  — >  V*  by 

M  =  -  A0,  0  e  V, 

owz 

where  as  usual  derivatives  are  interpreted  in  the  weak  or  distributional  sense.  We  may  now 
write  (2.13)  in  the  following  abstract  form 

v(t)  =  Av(t)  +  Be(t),  u(0)  =  v0,  (3.1) 

where  v(t)  is  used  as  the  shorthand  notation  for  the  function  v(t,  •)  (this  shorthand  notation 
will  be  used  throughout  the  remainder  of  this  section),  and  the  operator  B  :  MA/  — >  V*  is 
defined  by 

B£{w)  =  A b(w%  for  all  f  e  Mm. 

In  the  particular  case  studied  here  we  actually  have  b  e  L2  so  that  B  :  K07  — >  L2  C  V*.  It 
is  easy  to  argue  that  the  adjoint  B*  of  B  is  given  by 

B*(f)  =  (<6i,  0),  ( b2 ,  0),  •  •  • ,  ( bM ,  0))T  ,  for  all  06  k. 


Using  integration  by  parts,  we  obtain 


(A<j),'ip)v*,v 


W 

f  (  <920H  xu  w 

/  [  V~diA - X(P\W)  j  W\w)dw 

W 


d<j)(w ) 
dw  dw 


dw  — 


W 


W 


W 
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where  {-,-}v*y  denotes  the  usual  duality  product  [15,  36].  We  then  define  a  sesqnilinear 
form  a  on  V  x  V  by  a(0,  ifr)  =  {—A<f>,'ip)v*y,  that  is, 

W  W 

a(0,0)  =  [  ri9^  dw  +  [  A0(w)0(w)dw.  (3.2) 

J  aw  aw  J 

w  w 

We  see  immediately  that  a  is  symmetric,  and  hence  the  adjoint  A*  of  A  defined  by  a(0,  0)  = 
((J),  —A*'ijj)vy*  is  equal  to  A.  Now  we  may  rewrite  (3.1)  in  weak  form  as:  v(t)  G  V  for  all  t 
is  the  solution  of 


{v{t),  ^)v*y  +  a(v(t),  0)  =  ( Be(t),'ijj)v*y ,  u(0)  =  v0,  (3.3) 

for  all  -0  G  V.  By  (3.2)  and  Poincare’s  inequality,  we  find  that  there  exists  some  positive 
constant  c  such  that 

|a(0,0)|  <  (cA  +  r/)||0||y||0||y, 
holds  for  any  0, 0  G  V.  Similarly  for  all  0  G  V 

a(0,  0)  =  vUWv  +  AII0II h  >  vUWv- 

One  can  then  establish  estimates  and  arguments  as  in  [7,  8]  to  argue  that  A  generates  an 
analytic  semigroup  on  H,  V  and  V*.  Furthermore,  this  semigroup  is  exponentially  stable  on 
H,  V  and  V*. 

Turning  next  to  the  control  problem  for  the  abstract  dynamics  (3.1),  we  find  that  the  above 
results  along  with  Theorem  3.4  in  [8]  implies  that  the  algebraic  Riccati  equation 

(.4*11  +  IL4  -  IL6/T1 lB*U  +  J)0  =  0  for  all  0  G  V  (3.4) 

has  a  unique  nonnegative  solution  If  G  Jf(V*,  V )  and 

A  -  Bp-lB*n 

generates  an  exponentially  stable  semigroup  on  H,  V  and  V* .  Moreover,  the  optimal  feed¬ 
back  solution  that  minimizes  cost  functional  (2.15)  subject  to  (3.1)  is  given  by 

e0pt(*)  = 


3.2  Numerical  Approximation 

Our  goal  in  this  section  is  to  present  computational  methods  for  solution  of  the  feedback 
control  systems  under  investigation  here.  We  do  this  in  the  context  of  the  abstract  formu¬ 
lation  developed  in  [7,  8,  15]  and  summarized  above.  We  briefly  outline  a  method  based  on 
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a  standard  finite  element  approach.  For  the  convenience,  we  use  (')  for  the  derivative  with 
respect  to  the  space  variable  w. 


We  define  the  mesh  points  w )v ,  j  —  0  . . .  N  +  1  as  =  w,  w ^  =  w^_ ±  +  h  for  j  =  1, ...  N, 

UJ 

— .  Next  we  let  VN  be  a  sequence  of  finite  dimensional 


N 

w  —  w 


and  w%  ,1  =  w  where  h  = 

N+1  N  +  l 


subspaces  of  V.  In  particular,  VN  =  spa ,n{l^(w),  l™ (w), . . . ,  l^(w)}  C  V  =  H(\  (w,  w)  where 
the  piecewise  linear  basis  elements  {lf(w)}  are  defined  as  follows  for  j  —  1, . . . ,  N: 


w 


w 


N 

j~  1 


lf(w)  =  { 


W 


N 
3+ 1 


W 


,N 

°j- 1  - 


JN 

J3 


N  N 

“j-  1  -  Wj 


wf  <w  <  wf+1 


otherwise. 


(3.5) 


We  next  define  the  operator  AN  :  VN  — >  f/,v  (which  approximates  .4.)  by  restriction  of  a  to 
VN  x  VN]  this  yields 


(— A =  a(0,^),  for  all  (p,ip  G 

For  given  0  :  Mm  — ■>  I/*,  we  define  its  approximation  BN  :  Mm  — »  VN  by 

(BN£,  ip)  =  (f ,  FV>*  for  all  £  G  Rm  and  ^  G 

For  this  family  of  approximations,  the  corresponding  TVth  approximate  problem  in  VN  entails 
the  minimization  of  the  cost  functional 


JiV(e)  =  J  j  \vN(t,  w)\2dwdt  +  J  /3\e(t)\2dt, 

0  w  0 


(3.6) 


subject  to 

AA  =  ANvN(t)  +  vN(0)  =  VNv„.  (3.7) 

Here  vN  (t)  is  the  notation  for  vN(t,  •),  and  the  operator  denotes  the  usual  orthogonal 
projection  of  H  onto  VN .  That  is,  for  <p  G  H ,  we  have  VN (j)  G  is  defined  by 


(VN(p,'ip)  =  (0, -0),  for  all  ip  G 


(3.8) 


The  weak  form  of  (3.7),  i.e.,  the  approximate  problem  corresponding  to  (3.3),  can  then  be 
formulated  as  finding  vN(t)  G  VN  which  satisfies 


dvN(t ) 


'  dt 
vN(0)  =  VNv0. 


,ip)  +  a(v  (t),ip)  =  {B  e(t),ip),  ip  G  V  , 


(3.9) 
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It  is  well-known  [11]  that  for  any  <f>  G  V,  there  exist  a  sequence  <f>N  G  VN  such  that  \<pN  — 
<j> \v  — >  0  as  N  — >  oo.  Thus  we  can  be  assured  that  these  approximations  vN(t )  will  approach 
v(t)  for  N  sufficiently  large.  To  obtain  the  matrix  representations  for  the  operators  AN  and 
Bn  in  terms  of  the  piecewise  linear  spline  basis,  we  substitute 

N 

v(t)  ^VN(t)  = 

3= 1 

into  (3.9)  and  let  ijj  —  if  for  i  —  1,  2, . . . ,  N.  We  obtain  the  vector  system 


N 

3= 1 


(t)  (If,  li)  +  V 


N 

3= 1 


N 

Ty >  )0 + ^  ^2  uj 

3  =  1 


(l)(LVf)=AVq(l)(6,lf) 


M 


fe=i 


We  note  that  the  above  equation  can  be  written  in  the  matrix  form 

Fnvn  +  VQN  vn  +  \FNvN  =  XGNe,  (3.10) 

where  vN(t)  =  (v3  (t),  zyf (t), . . . ,  v^(t))T ,  FN  and  QN  are  N  x  N  matrices  with  their  (i,  j)th 
elements  defined  by 


if  i=j 

(  2 

h' 

if  i  =  j 

h 

if  \i-j\  =  1 

and  ((/f )',  (/f )')  =  < 

1 

~h' 

if  \i  —  j\  =  1 

0, 

otherwise, 

.  0, 

otherwise, 

respectively,  and  GN  is  an  N  x  M  matrix  with  its  (i,j)th  element  being  defined  by  (bj,  if). 
Note  that  (3.10)  can  be  simplified  as  follows 

vN(t)  =  ANuN(t )  +  BNe(t),  (3.11) 


where 

AN  =  _(FNyl(vQN  +  XFN^  BN  =  X  ( pNy 1  qN, 

are  the  matrix  representations  for  operators  AN  and  BN ,  respectively.  We  consider  the 
approximation  1^(0)  to  the  initial  condition  u(0).  To  do  this,  we  substitute 

N 

v(0)x:vN(0)=PNv„  =  J2‘'oJ? 

3= 1 

into  (3.8)  with  <j)  =  u0,  and  let  ^  =  if  for  i  —  1,  2, . . . ,  N,  and  we  find 

N 

lf)  =  (t;0,  li),  i  —  1,2, ,  N. 

3  =  1 
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Let  Va  = 


.N  /.N 


'0,1) 


•  •  5  v on )T ■  Then  from  the  above  equation  we  have 

(  (vo,  l?)  \ 

(vo  ,1?) 


^  =  ( F r 


V  (vo  Jn)  j 


(3.12) 


We  now  solve  (3.6)  subject  to  (3.7)  to  obtain  approximations  to  the  optimal  eGp^  denoted  by 
e^pt .  Note  that  the  injection  from  V  to  H  is  compact.  Hence,  by  Theorem  4.8  in  [8],  for  N 

sufficiently  large,  there  exists  a  unique  nonnegative  self-adjoint  solution  nA  to  the  algebraic 
Riccati  equation  in  VN 

(An)*un  +  nv^v  _  n  NBNp-1(BNynN  + 1  =  o,  (3.13) 


and  the  convergence  of  the  Riccati  and  control  operators  are  also  obtained.  In  addition,  the 
feedback  system  operator  A  —  B(3~1(BN)*T1N  (i.e.,  the  approximate  feedback  controls  used  in 
the  original  infinite  dimensional  system)  generates  an  exponential  stable  analytic  semigroup 
on  H  and  for  vq  €  H 


l^opt)-^opt)l<7(iV)||w||i 


where  e£pt(t)  = 
sentation,  e^p^. 


=  --(BN)*UNvN(t),  and  'y(N)  ^  0  as  N 
P 

is  given  by 


(BN)T7TNVN(t), 


oo. 


In  terms  of  matrix  repre- 


where  nN  is  an  N  x  N  matrix  representation  of  the  operator  njV  given  by  the  corresponding 
matrix  representation  for  algebraic  Riccati  equation  (3.13),  i.  e., 


(AN)TnN  +  nNAN  -  7 TNBNp-\BN)T7rN  +  1  =  0. 


We  can  easily  solve  for  nN  using  the  built  in  MATLAB  function  are  or  other  available 
software. 


We  have  thus  gathered  all  of  the  information  needed  to  solve  for  vN .  At  this  time  recall 
equation  (3.11)  with  the  optimal  e^p^  is  given  by 

uN(t)  =  {AN  -  -BN(BN)TnN}uN(t) 

P 

with  initial  conditions  uN(0)  =  Uq  defined  by  (3.12).  We  use  these  approximations  in  the 
numerical  results  presented  below. 
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4  Numerical  Results 


In  this  section,  a  number  of  simulations  are  carried  out  to  investigate  the  proposed  approach 
and  illustrate  its  usefulness  in  exploring  questions  of  control  in  a  dynamic  player  game 
study.  All  of  the  computational  results  presented  in  this  section  are  obtained  with  rj  —  6 
and  A  =  1.  We  use  the  piecewise  linear  approximations  presented  in  the  previous  section. 
In  the  examples  given  here,  we  typically  used  iV  =  50  basis  elements.  The  number  of  basis 
elements  used  was  arrived  at  after  simulations  studies  to  ascertain  values  of  N  required  to 
insure  convergence. 


For  the  intensity  function  r(w,  e)  of  (2.7)  we  used  the  magnitude  of  Fresnel  reflection  coeffi¬ 
cient  based  on  planar  layer  geometry  [6]  given  by  r(w,  e)  —  ^  + 


1  +  ab 


,  where 


a  =  6  and  b  =  e^V^wd/c  (4T) 

e  +  yfejl 

with  magnetic  layer  permeability  fi  —  1,  layer  thickness  d  =  2.5  mm  and  speed  of  light 
c  =  3  x  108  m/sec.  We  chose  S  =  {e*}fL1  by  taking  M  —  50  equal  partition  points  e*  in  the 
dielectric  permittivity  interval  1  <  e  <  1000,  and  interrogator  frequency  range  [re,  w]  with 
w  =  .4  GHz  and  w  —  1  GHz. 


4.1  Single  Carrier  Frequency  Input 

Our  design  parameter  is  chosen  to  be  (3  =  0.25  for  all  the  results  obtained  in  this  part.  Our 
first  attempt  for  the  initial  condition  v$  is  a  truncated  normal  (Gaussian)  distribution.  The 
plots  for  the  numerical  approximations  of  vN(t,w )  and  the  control  uN(t,w )  are  illustrated 
in  Figure  1.  This  reveals  that  when  the  most  emphasis  in  interrogating  frequencies  is  placed 
on  frequencies  around  .7  GHz,  that  the  best  controls  are  ones  corresponding  to  materials 
effective  around  the  frequency  .7  GHz. 

In  the  next  two  trials,  we  choose  distributions  concentrated  at  the  extreme  frequencies  for 
an  initial  choice  for  v$.  Specifically,  the  numerical  results  illustrated  in  Figure  2  are  obtained 
with  a  truncated  Gamma  distribution.  We  see  from  Figure  2  that  when  the  most  emphasis 
is  placed  on  frequencies  around  .4  GHz,  that  the  best  controls  (materials)  are  ones  effective 
around  the  frequency  .4  GHz  as  well  as  some  measure  of  control  is  exhibited  at  frequencies 
around  1  GHz.  This  agrees  with  what  was  observed  in  [6]  in  that  a  material  that  nulls  well 
at  .4  GHz  also  has  some  ability  to  null  at  1  GHz. 

Figure  3  illustrates  the  numerical  results  obtained  with  a  truncated  Beta  input.  From  this 
figure  we  see  that  when  the  most  emphasis  is  placed  on  frequencies  around  1  GHz,  that  the 
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Figure  1:  Numerical  results  obtained  with  initial  condition  Vo  given  by  a  truncated  normal 
distribution,  (left):  vN(t,w);  (right):  uN(t,w). 


Figure  2:  Numerical  results  obtained  with  initial  condition  vq  given  by  a  truncated  Gamma 
distribution,  (left):  vN(t,w );  (right):  uN(t,w). 


x10~6 


xIO'6 


5-, 


Figure  3:  Numerical  results  obtained  with  initial  condition  u0  given  by  a  truncated  Beta 
distribution,  (left):  vN(t,w );  (right):  uN(t,w). 
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best  materials  to  use  are  ones  effective  around  the  frequency  1  GHz.  This  illustrates  what 
was  seen  in  the  static  examples  of  [6]  with  a  material  that  cloaks  well  at  1GHz. 


4.2  Multiple  Carrier  Frequency  Inputs 

In  this  section,  the  numerical  results  are  obtained  with  [3  =  0.25.  The  input  is  chosen  from 
different  distributions  in  a  sequence  of  interrogating  pulses.  That  is,  the  interrogator  uses  one 
distribution  which  is  unknown  to  the  evader  to  sample  for  interrogating  pulse,  then  switches 
to  choosing  from  a  second  distribution  and  then  a  third.  This  is  done  in  an  effort  to  confuse 
the  evader  in  his  choice  of  surface  permittivities.  These  simulations  are  thus  a  rudimentary 
example  of  the  situation  where  the  interrogator  also  has  a  (non-feedback)  time  dependent 
control  for  the  input  frequency  distributions.  We  simulate  this  by  presenting  graphs  of 
responses  to  a  sequence  of  consecutive  initial  condition  inputs  v0.  As  we  can  see  from 
a  variety  of  different  combinations  in  Figures  4-6,  the  computational  results  demonstrate 
that  the  evader  control  quickly  switches  in  time  to  accommodate  the  new  choices  in  the 
interrogator  frequency  distributions.  This  suggests  some  level  of  robustness  in  the  evader’s 
response  to  changing  interrogator  frequencies. 


Figure  4:  Numerical  results  obtained  with  a  sequence  of  consecutive  initial  conditions  Vq 
given  by  truncated  normal-truncated  Gamma-truncated  Beta  distributions,  (left):  vN(t,  w); 
(right):  uN(t,w). 
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Figure  5:  Numerical  results  obtained  with  a  sequence  of  consecutive  initial  conditions  v0  given 
by  truncated  normal-truncated  normal-truncated  normal  distributions,  (left):  vN(t,w ); 
(right):  uN(t,w). 


Figure  6:  Numerical  results  obtained  with  a  sequence  of  consecutive  initial  conditions  Vq 
given  by  truncated  Beta-truncated  Gamma-truncated  Beta  distributions,  (left):  vN(t,w ); 
(right):  uN(t,w). 


4.3  Effect  of  Design  Parameter  on  the  Overall  Control 

Finally,  we  consider  how  the  choice  of  the  design  parameter  /3  affects  the  overall  control 
effectiveness.  All  the  numerical  results  in  this  section  are  obtained  with  a  truncated  normal 
initial  condition  u0-  We  note  from  Figures  7-9  as  (3  increases  (i.e.,  the  control  gain  decreases) 
the  feedback  control  action  is  less  effective  and  rapid. 
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Figure  7:  Numerical  results  obtained  with  (3  —  2.5  x  10  4.  (left):  vN(t,w );  (right):  uN(t,w). 


Figure  8:  Numerical  results  obtained  with  f3  =  0.25.  (left):  vN(t,w );  (right):  uN(t,w). 


Figure  9:  Numerical  results  obtained  with  [3  =  25000.  (left):  vN(t,w );  (right):  uN(t,w). 
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5  Conclusion  and  Future  Research  Efforts 


In  this  paper,  we  consider  a  dynamic  evasion-interrogation  games  with  uncertainty  in  the 
context  of  electromagnetics.  Two  different  formulations  are  considered:  one  is  based  on  the 
probability  density  function  of  the  intensity  of  the  reflected  signal,  and  the  other  is  based  on 
the  expected  value  of  the  intensity  of  the  reflected  signal.  We  should  note  that  we  anticipate 
that  the  ideas  presented  here  can  be  readily  implemented  in  a  number  of  other  modern 
non-cooperative  adversarial  situations  such  as  information  warfare  and  network  security. 

There  are  several  efforts  we  plan  to  pursue  in  the  near  future.  One  is  to  extend  the  ideas  in 
this  paper  to  include  a  stochastic  process  for  the  evader  to  obtain  a  true  two  player  min-max 
dynamic  differential  game  for  the  evader-interrogator  problem.  The  other  efforts  include 
investigation  of  other  means  to  introduce  uncertainty  in  the  dynamic  two  player  games. 
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